Guideline: In More Details - Iterative And Agile System Development

In these forms of system development, an important question is who is responsible for a test level. This is clear for unit and unit integration tests, but there are a number of options for the system and acceptance tests. The table below lists the most common test levels and their responsible parties.

‘In or outside of team’ means that a test can be executed within the project team, but that for some test levels, a separate test team outside the project team or even outside the organisation can be created to execute the test. For instance, the production acceptance test is usually executed outside the project team. This test requires specific (system management) expertise on the testers’ part, plus a test environment that is as faithful to the production environment as possible. Often, both are available within the project on a limited basis.

The strong involvement of users with a decision-making mandate, and the collaboration within the team, make that the traditional wall between developers and users/acceptants is much lower. In the course of the process, users provide continuous feedback on prototypes (by means of testing and evaluation), due to which the acceptance test can be a very ‘thin’ test. This will consist primarily of tests that could not be executed earlier, such as an end-to-end test. If the acceptance test is executed in the project, one may then also consider integrating it with the system test. The test team then comprises a mix of users, developers and professional testers.

Less logical is executing the acceptance test outside the project. This is not really in line with the principles of iterative development. The choice depends on the risks. If the organisation expects high risks, an acceptance test outside the team is more logical even though it is not in line with iterative principles. Examples of such risks are an insufficient decision-making mandate for users, that the project manager or developers seem to ‘dominate’ the users, or that the system is so critical that testing must occur in an adequately controlled, stable and production-like environment.